Load Testing, Scaling and Monitoring the Applications

Now let’s add the CPU metrics of the Micronaut and Spring apps to the same chart. Select Add metric and select "CPU Usage" and aggregation "Max" again. Then click on the Scope setting, unselect quarkus-app and pick the micronaut-app. Select Apply, and repeat the same for the spring-app.

Now you should see a nice chart with the CPU usage of all three applications. Let’s save this chart on the dashboard!

Select Save to dashboard and choose Pin to dashboard:

In the Pin to dashboard dialog, select Create new and give it a name, for example "Java Runtimes" then click Create and pin.

We’ll also add charts to monitor the memory usage and number of replicas of our applications.

Click on +New chart (top left corner) and repeat the same steps as before to create a chart with "Memory Working Set Bytes" metrics with aggregation "Max". Do this for the three applications (Quarkus, Micronaut and Spring Boot). Select Save to dashboard, choose Pin to dashboard, and select the existing "Java Runtimes" dashboard.

Click on +New chart again, and create another chart with the "Replica Count" metric with aggregation "Max". Don’t forget to save this dashboard as well (select Save to dashboard, choose Pin to dashboard, and select the existing "Java Runtimes" dashboard).

When you’re finished, in the Azure Portal sidebar select Dashboards and choose the "Java Runtimes" dashboard we created earlier.

You should now see the 3 charts you just created. You can rearrange the charts by dragging them around, and resize them by dragging the bottom right corner.

The most straightforward way to access the logs of your application is to connect to a given container instance and get the stream of console logs. You can directly access the stream of logs from the latest revision of a running instance of your application with this command:

Once you’re connected, you can see the logs of your application in real time. The --follow option keeps the connection open to see the new logs in real time.

In a different terminal, you can use the curl command to generate some load on the application. You should see the logs of the application being updated in real time.

You can also access the console logs in the Azure Portal or via CLI. Using Log Analytics, you get access to all logs from all your applications, revisions, and instances. This means you can troubleshoot issues across all your applications, tracing requests as needed, which is crucial if your application use a microservices architecture.

Open the Azure Portal and navigate to the resource group rg-java-runtimes you created for your application. Select the logs-java-runtimes container app, then select Logs from the left menu, under the General group.

By default, you are presented a list of pre-defined queries. Close this panel by clicking on the X button on the top right corner, as we’ll create our own query.

Log analytics queries use the Kusto query language, which is a SQL-like language. You can use the query language to filter the logs and get the information you need.

Let’s start by creating a query to get the logs of the quarkus-app container app. Enter this query in the editor:

You can get the app revision name by running the following command:

Select Run to execute the query. You should see the logs of the quarkus-app container app.

For now, it’s not very useful as it’s the same logs we saw in the previous section. Let’s add some filters to the query to search for error messages from the logs:

Select Run to execute the query. If your application is working fine, you should not see any results. Let’s generate some errors by crashing the application with the following command:

Oops! We’re trying to allocate more memory than the container has available, resulting in a crash because of our (crude) memory allocation algorithm.

If you run the query again, you should see the error message in the logs:

We can make it easier to read by making the latest logs appear at the top of the results, and only show the time and message of the last 10 logs:

And here we can quickly see our OutOfMemoryError error message.

We can save the query for later use by clicking on the Save button. You can then give it a name and a description, and a category to quickly find it later.

You can also view the logs via a CLI command instead of using the portal. It’s just a matter of writing any KQL query in the analytics-query parameter as follow:

While console logs are very useful to troubleshoot issues with your application, it won’t be much help if your container is not running. In this case, you need to troubleshoot the container host itself. This is where the system logs come in handy.

The system logs are the logs of the container host. They are very useful to troubleshoot issues with the container host itself and monitor its activity during provisioning operations.

System logs can be accessed the same way as console logs. Change the previous query in the editor to get the system logs:

You should see the system logs of the quarkus-app container app, covering the whole lifecycle of the container.

Using the CLI you can execute:

To use Azure Load Testing, go to the Azure Portal, select Create a resource in the sidebar and search for "Azure Load Testing".

1. Select Create:

2. In the Resource group field, select the rg-java-runtimes that we created previously.

3. Set the name lt-java-runtimes-<Your unique identifier> for the load testing instance (the name has to be unique).

4. Set the location to match our previously created resources (East US).

5. Select Review + Create, then Create.

Creating a load testing resource can take a few moment. Once created, you should see the Azure Load Testing available in your resource group:

Select lt-java-runtimes, and then click on Tests and then Create.

1. You can either create a quick load test using a wizard, or create a load test using a JMeter script. Choose this second option.

2. Before uploading a JMeter script, create a load test by entering a name (eg. "Add some load"), a description and click on the "Test plan" tab:

3. Now that you are on the "Test plan" tab, you can upload the JMeter file (located under scripts/jmeter/src/test/jmeter/load.jmx) as well as the user.properties file. The JMeter file sets up a load campaign targeting the various endpoints.

But before uploading the user.properties file, make sure you change the properties (QUARKUS_HOST, SPRING_HOST and MICRONAUT_HOST) to match your deployed applications:

4. Select Upload, and choose "User properties" in the File relevance field of the user.properties file.

5. Select Review + Create, then Create.

After the test creation, it will start automatically after a short time. When the test run finishes, you will get some metrics:

In the Azure Portal sidebar, select Dashboards and go back to the "Java Runtimes" dashboard we created earlier.

If you take a look at the charts, you can see CPU and memory usage increase, and also that the number of replicas has increased from 1 replica to 10. Azure Container Apps has scaled automatically the application depending on the load.

To scale based on CPU usage, we need to update the scale rule of the application to use the cpu trigger. This will create a new revision of the application (but the URL $QUARKUS_HOST remains unchanged), and will start a new deployment.

We will set a new scale rule for our Quarkus app using the Azure CLI:

This will automatically scale out the application when the CPU usage is above 10% (we set it low deliberately to make it easy to go up).

Go back to the Azure Portal, and search for lt-java-runtimes to open again our load testing instance. Select Tests in the left sidebar, open the test we created earlier and select Run to run the load tests again.

Once the test is finished, go back to the "Java Runtimes" dashboard and take a look at the number of replicas chart again. You should see that the number of replicas has increased to 10, and that the CPU usage has increased as well.

Another option that we can use is to scale based on the memory usage, with the memory trigger.

This we will set the scale rule for our Micronaut app using the command:

This will automatically scale out the application when the memory usage is above 15% (we set it low deliberately to make it easy to go up).

Again, go back to the Azure Portal and run the load tests again. Open the dashboard when the test is finished, and take a look at the number of replicas chart again.

You can now compare how the CPU (Quarkus), memory (Micronaut), and HTTP (Spring) triggers behave when scaling the application, under the same load.

As you can see, using different scaling triggers allows to tune the scaling behavior of your application, depending on the type of load you want to handle. Note that you’re not limited to only one scaling trigger, you can use multiple triggers at the same time.